System and Method for Fast Variable-Size Motion Estimation

ABSTRACT

Methods and systems for fast variable block-size motion estimation based on merging and splitting procedures for AVC video encoding are disclosed. The methods take advantage of the correlation of the Motion Vectors (MVs) of the different block-size modes to achieve a good computation reduction. Considering that the smaller the block-size difference between two block-sizes is, the more correlations between their MVs can be expected, the methods use a 8*8 block as an initial processing block to calculate prediction MVs. The prediction MVs are then used in predicting MVs for other block-sizes.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional application of application Ser. No.10/986,649, filed Nov. 12, 2004, incorporated herein by reference in itsentirety, and claims the benefit of its earlier filing date under 35U.S.C. 120(e).

FIELD OF THE INVENTION

The present invention relates generally to a method and system ofvariable block-size motion estimation and more particularly, to a methodand system of fast variable block-size motion estimation based on mergeand split procedures for H.264/MPEG-4 AVC video encoding.

BACKGROUND OF THE INVENTION

In current video coding standards, Motion Estimation (ME) is a corefunctional block to remove temporal redundancy in video sequences toachieve high compression. In the emerging H.264/MPEG-4 AVC video codingstandard, for Inter coded Macroblocks (MBs), tree-structured block-sizescan be employed in the motion estimation. Each 16*16 MB can be coded in16*16, 16*8, 8*16, and 8*8 block-modes. If the 8*8 block-mode is chosen,each 8*8 sub-macroblock can be independently partitioned into 8*8, 8*4,4*8, and 4*4 blocks. So, altogether there can be seven differentblock-sizes: 16*16, 16*8, 8*16, 8*8, 8*4, 4*8, and 4*4. For theseblock-sizes, each 16*16 MB contains 1, 2, 2, 4, 8, 8, and 16 blocksrespectively.

In the H.264/MPEG-4 AVC reference software, a Fast Full Search (FFS)algorithm is used for the ME. The SADs (Sum of Absolute Differences) of4*4 blocks are first calculated and the SADs of the other sixblock-modes are calculated by summing up the SADs of the corresponding4*4 blocks. Although the SADs in the calculation of the 4*4 block-modeare re-used, for all seven block-sizes, the SAD calculation load will belarger than that using a full-search ME for a 16*16 MB. For example, ifa search-window size is 31*31 pixels and the Unrestricted Motion Vector(UMV) mode is enabled, the number of search points using the full-searchME is 31*31=961. Accordingly, the required SAD computation is largerthan that for a 16*16 MB with 961 search points, which requires a lot ofcomputations.

Recently some fast variable block-size ME algorithms have been proposed.For example, a fast search algorithm is applied to the seven block-sizesindependently, as disclosed by Ma et al. in an article entitled “AnImproved Adaptive Rood Pattern Search For Fast Block-matching MotionEstimation in JVT/H.26L”. A merge procedure is also proposed, asdisclosed by Tu et al. in “Fast Variable-size Block Motion EstimationUsing Merging Procedure With an Adaptive Threshold”, which determinesthe MVs of larger block-sizes from the MVs of smaller block-sizes, withthe threshold for the merge criteria related to the quantizationparameter. Kucukgoz et al. in their article entitled “Early-stop andMotion Vector Re-using for MPEG-2 to H.264 Transcoding” apply abottom-up merge scheme and an early-stop strategy for the variableblock-size ME in MPEG-2 to H.264 transcoding. Furthermore, in “FastInteger Pixel Motion Estimation” disclosed by Li et al., a merge andsplit process for the ME is proposed. The disclosure, however, fails todescribe the detailed process.

BRIEF SUMMARY OF THE INVENTION

Accordingly, one object of the present invention is to provide a MVprediction and MV search strategy for fast variable block-size MEs.Using the strategy, fast variable block-size ME algorithms based onmerge and split procedures are proposed and described in details.Experimental results show that using these merge/split schemes, thesearch-points can be reduced to about 4% of that using the fastfull-search ME for a 16*16 MB, with negligible quality degradation.

One embodiment of the present invention is to provide method forpredicting motion vectors (MVs) of variable block sizes of a video imageframe. The method includes choosing a block size as an initial processedblock, wherein the initial processed block has a block size between asmallest block size a largest block size of the variable block sizes,partitioning the initial processed block into a plurality of sub-blocksand obtaining a plurality of prediction MVs from the plurality ofsub-blocks. The plurality of prediction MVs are then used in subsequentsteps. The initial processed block is merged into larger-size blocks andthe plurality of prediction MVs are used in the merging process topredict MVs of the larger-size blocks. Further, the initial processedblock is split into smaller-size blocks; and similarly, the plurality ofMVs are used in the splitting process to predict MVs of the smaller-sizeblocks.

According to the present invention, the initial block size is 8*8. Inthis case, the initial block is divided into four 4*4 sub-blocks andfour prediction MVs: MV.sub.A, MV.sub.B, MV.sub.C, and MV.sub.D arecalculated. The initial block is merged into 8*16, 16*8, and 16*16, andis split into 8*4, 4*8, and 4*4 blocks.

In the first preferred embodiment, if any two prediction MVs matcheswith each other, the matched MV is used as a prediction MV of nextlarger-size block. If none of the prediction MVs matches with eachother, the method searches for a new MV for the larger-size block.Similarly, if the MVs of the smaller-size blocks, such as 8*4, and 4*8blocks, are the same, the MV is used as a prediction block of the 4*4block. Otherwise, the prediction MVs of the four sub-blocks are averageand the averaged MV is used as a search center.

A second embodiment of the present invention is to provide a method forjudging motion vector (MV) consistency of a block in a video imageframe. The method includes dividing the block into a plurality ofsub-blocks, calculating prediction MVs of the plurality of sub-blocks,comparing the prediction MVs of the plurality of sub-blocks, and ifhorizontal components and vertical components of any adjacent two of theplurality of sub-blocks MV_(A) and MV_(B) match with each other, i.e.,|MV_(A-)x−MV_(B-)x|==0 and |MV_(A-)y−MV_(B-)y|=0, then prediction MV_(a)is used as a reference MV for a larger-size block.

In the above embodiment, if the horizontal components and verticalcomponents of any adjacent two of the plurality of sub-blocks MV_(A) andMV_(B) do not match, then the method searches MVs for a larger-sizeblock, in which the criterion for unmatched MV is:

|MV _(A-) x−MV _(B-) x|>=3 or |MV _(A-) y−MV _(B-) y|>=3.

A third embodiment of the present invention is to provide a systemmodule for fast motion estimation of variable block sizes of a videoimage frame. The system includes a first motion estimation sub-modulefor processing an initial n×m block to obtain a plurality of predictionMVs and a merging sub-module for merging the n×m block into larger-sizeblocks and for receiving the plurality of prediction MVs from the firstmotion estimation sub-module. The merging sub-module uses the pluralityof prediction MVs to calculate prediction MVs for the larger-sizeblocks. The system also includes a splitting sub-module for splittingthe n×m block into smaller-size blocks and for receiving the pluralityof prediction MVs from the first motion estimation sub-module. Thesplitting sub-module uses the plurality of prediction MVs to calculateprediction MVs for the smaller-size blocks.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart showing fast variable block-size motionestimation in accordance with the present invention.

FIG. 2 is a schematic diagram showing an 8*8 block merging into 8*16,16*8, and 16*16 blocks.

FIG. 3 is a schematic diagram showing an 8*8 block splitting into 8*4,4*8, and 4*4 blocks.

FIG. 4 is a schematic diagram showing a SDSP MV search.

FIG. 5 is a schematic diagram of a system module for fast variableblock-size motion estimation in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Generally, a motion estimation algorithm considers two importantcomponents for searching the MV: search center and search pattern.Conventionally, the search center is generally predicted by taking amedian value, a mean value, or that with minimum SAD from the spatialand temporal neighbors' MVs. However, this type of methods is based onthe assumption that the MV field is homogeneous. In the case of a videosequence with rich local motions and relatively small moving objects,the assumption is often not valid.

Since all the different-size blocks are inside the same MB (i.e., MotionBlock), it is expected that the MVs of these blocks have strongcorrelations. Therefore, in accordance with the present invention, itassumes that the prediction result will be better than that from itsneighboring MBs if the search center of the current block is predictedfrom the MVs of its sub-blocks or its upper-layer blocks, especiallywhen the motion field is not homogeneous.

To accurately obtain the correlation of different block-sizes, theinitial block-size and the accuracy of the MV prediction are important.Intuitively, two methods can be employed: bottom-up merge and top-downsplit. In the bottom-up merge, a 4*4 block is taken as the initialblock-size. Since the 4*4 block-size is relatively small, its MV caneasily point to a best match but that does not necessarily represent thetrue motion. Thus, the MVs may not be reliable. In the top-down split, a16*16 block is taken as the initial block-size. Due to the relativelylarge block-size, however, there is a higher possibility that the motioninside the block is not homogeneous. Therefore, the MVs of thesub-blocks can be different from each other and from the MV of the 16*16block. In this case, it may take a longer process to find the MV of thesub-blocks.

Another consideration is that the smaller the block-size differencebetween two block-sizes is, the more correlations between their MVs canbe expected. Therefore, according to the present invention, a reasonableapproach is to take an 8*8 block as the initial block-size, since theblock-size of 8*8 is relatively close to both the block-sizes of 16*16and 4*4. Using an 8*8 block as the initial block-size has an additionaladvantage. Due to the fact that at low bit-rates, larger block-sizes arepreferred in general since smaller block-sizes result in more MVoverheads and at high bit-rates, small block-sizes are preferred ingeneral since the advantage from the better predictions out-weights theMV overheads. The 8*8 block is a suitable initial block-size regardlessof the bit-rates.

Accordingly, a fast ME scheme by Merge and Split, which starts from the8*8 block-size, and merges to larger block-sizes or splits to smallerblock-sizes is disclosed as shown in a preferred embodiment of FIG. 1.

In FIG. 1, a merge process 10 is first performed. That is, 8*8 blocksare merged into 8*16, 16*8, and 16*16 blocks. To illustrate the mergingprocess, please further refers to FIG. 2 that shows 8*8 blocks 21 can bemerged into 8*16 blocks 22, 16*8 blocks 23, and 16*16 blocks 24.

At step 101, an ADSS (Adaptive Diversity Search Strategy) ME is firstperformed for the 8*8 blocks 21 to search for its MVs. Since the MVs ofthe 8*8 blocks will be used for prediction in the merge and splitprocess, the overall results are highly dependent on the accuracy of theMVs. The ADSS is used in this step since it can achieve a good PSNR(Peak Signal-to-Noise Ratio) with a reduced number of search points.

As shown in FIG. 2, the 8*8 block is divided into four sub-blocks A, B,C, and D, and the prediction MV of each of the sub-blocks is MV_(A),MV_(B), MV_(C), and MV_(D). In the merging process, sub-blocks A and Bcan be merged to sub-block E in the 16*8 block. Similarly, sub-block Aand C can be merged to sub-block G. Therefore, if adjacent MVs, such asMV_(A) and MV_(B), are matched, MV_(A) (or MV_(B)) is used as aprediction MV of sub-block E of a 16*8 larger block size. However, ifnone of the adjacent MVs matched with each other, the process thensearch the MVs for larger block-sizes. Otherwise, MV_(A), MV_(B),MV_(C), and MV_(D) are averaged and the average MV is used for search MVfor larger block sizes. The determination of matched MV, unmatched MVand averaging MV will be described in more detail later.

The match MV is determined at step 102 of FIG. 1 according to thefollowing equation:

|MV _(A-) x−MV _(B-) x|==0 and |MV _(A-) y−MV _(B-) y|==0.

wherein MV_(A-x) and MV_(A-Y) are horizontal component and verticalcomponents of prediction MV_(A) and MV_(B-.X) and MV_(B-Y) arehorizontal component and vertical component of prediction MV_(B),respectively. As described, when the horizontal and vertical componentsof MV_(A) and MV_(B) matches, MV_(A) will be considered as the MV of alarger block (i.e., 8*16, 16*8, and 16*16), as shown at step 103.

Again, at step 102, if none of sub-blocks A-D of the 8*8 blocks matcheswith each other, the process searches the MV for the larger block-sizeswith ADSS, as shown at step 104. According to the present invention, thecriterion for determining unmatched MV follows:

|MV _(A-) x−MV _(B-) x|>=3 or |MV _(A-) y−MV _(B-) y>=3.

At steps 105 and 106, for those search results other than matched MVs asshown at step 103 and unmatched MVs as shown at step 104, the processaverages MV as the prediction MV, and performs the MV search for largerblocks such as 8*16, 16*8, and 16*16 blocks. The process for steps 105and 106 are described as below.

The MV predictions for larger block sizes, i.e., 8*16 and 16*8 blockscan be performed as follows:

PMV _(E)=(MV _(A) +MV _(B))/2

PMV _(F)=(MV _(C) +MV _(D))/2

PMV _(G)=(MV _(A) +MV _(C))/2

PMV _(H)=(MV _(B) +MV _(D))/2  (1)

After the above MV predictions, a fast MV search procedure, which is asdescribed above with reference to steps 101-102 and 104-105, isperformed to find the MVs for the blocks E, F, G, and H. After the MVsfor blocks E, F, G, and H are obtained, the MV prediction for a 16*16 MBcan be calculated as follows:

PMV _(I)=(MV _(E) +MV _(F) +MV _(G) +MV _(H))/4  (2)

In addition, the present invention utilizes a Cumulative DistributionFunction (CDF) of “the delta MVs between the Prediction MVs and the MVsby Full-Search is less or equal to d pixels”, to fine-tune the accuracyof the MV estimation. The CDF is defined by Equation (3):

$\begin{matrix}{{P\left( {{delta\_ MV}<=d} \right)} = {P\left( {{\bigcup\limits_{i \in {\{{{16*8.8*16},{16*16}}\}}}\left( {{{PMV}_{i}{\_ x}} - {{MV}_{i}{\_ x}{ + }{PMV}_{i}{\_ y}} - {{MV}_{i}{\_ y}}} \right)}<=d} \right)}} & (3)\end{matrix}$

where PMV_(i-)x and PMV_(i-)y is the horizontal component and thevertical component of the prediction MV for block i respectively,MV_(i-)x and MV_(i-)y are MV components by the Full-Search, “∪”represents “or”, which includes all predicted modes (16*8, 8*16, and16*16) using Equation (1) and (2), and d is an integral no greater than3.

The CDF with up to three pixels (i.e., d=3) offset is listed in Table-1.d=0 means that the MVs by Full-Search and predicted MV by the presentinvention are the same. In this case, the average of the MVs of thesmaller-size blocks is taken as the MV for the larger-size blocks, whichis similar to that in a pure merge process, the accuracy is only 49%,49%, 68%, and 49% for the four sequences simulated, respectively.However, if the average of the MVs is taken as the prediction MV for thelarger size blocks, followed by a one-pixel SDSP (Small Diamond SearchPattern) search, the accuracy can be improved to 85%, 86%, 89% and 85%of the 16*8, 8*16 and 16*16 blocks, respectively. By three-steps of SDSP(that is, d=3 pixels), the accuracy rate can be as high as 97%, 98%, 96%and 94%. Similar conclusions can also be drawn from Table-2 and 3, whichshow CDF results for smaller blocks after a split process by using theSDSP. Tables-2 and 3 will be described later.

TABLE 1 CDF of MV difference between MVs by Full-Search and predictedMVs for 16*16, 16*8, and 8*16 blocks d = 0 d = 1 d = 2 d = 3 ForemanQCIF 49.44% 84.99% 93.50% 96.62% Coast Guard CIF 49.04% 86.18% 94.95%97.83% MAD CIF 68.00% 88.96% 93.97% 96.07% Mobile CIF 48.58% 85.41%91.48% 93.52% (Search range = [−16, 15], Fixed QP = 30, frame distance =1.)

Again refer to FIG. 1. After obtaining estimated MVs of the larger blocksizes, the estimated MVs of smaller block sizes can be performed bysplitting process 20. In this case, the 8*8 blocks are split into 8*4,4*8, and 4*4 block-sizes, as shown at step 105. The split from 8*8blocks 31 into 8*4 blocks 32 and 4*8 blocks 33 and then into 4*4 blocksis further illustrated in FIG. 3.

As shown at steps 107 and 108, splitting process 20 first takes the MVof the corresponding larger-block as the prediction MV and performs theSDSP search and searches MV for 8*4 and 4*8 blocks with the SDSP.

At step 109, if the two MVs of the corresponding 8*4 and 4*8 blocksequal to each other, then the MV is returned as the MV of the 4*4 block,as shown at step 110. Otherwise, the process calculates an average MV asa predicted MV, at step 111, and searches MV for the 4*4 block withSDSP, at step 112.

After MV for the 4*4 block is estimated, the splitting process iscompleted.

The CDF of MV difference between the MVs by Full-Search and thepredicted MVs for 8*4 and 4*8 blocks is shown in Table-2.

For splitting from 8*4 blocks and 4*8 blocks to 4*4 blocks, since thereare two larger blocks covering each 4*4 block, the average MV is takenfor prediction. The CDF of the MV difference is shown in Table-3.

TABLE 2 CDF of MV difference between MVs by Full Search and predictedMVs for 8*4, and 4*8 blocks d = 0 d = 1 d = 2 d = 3 Foreman QCIF 72.25%89.47% 95.27% 97.30% Coast Guard CIF 67.49% 89.37% 95.23% 97.00% MAD CIF82.37%q 92.42% 96.07% 97.48% Mobile CIF 68.60% 88.10% 92.84% 94.51%

TABLE 3 CDF of MV difference between MVs by Full Search and predictedMVs for 4*4 blocks d = 0 d = 1 d = 2 d = 3 Foreman QCIF 66.06% 92.38%96.92% 98.20% Coast Guard CIF 62.82% 92.40% 95.60% 96.90% MAD CIF 77.38%94.43% 97.04% 97.81% Mobile CIF 60.07% 86.55% 91.52% 93.71%

In accordance with the present invention, in addition to taking thepredicted MV from the simple merge or split process as the final MV, anMV search with SDSP is also applied after the MV prediction. FIG. 4shows a SDSP MV search pattern utilized in the present invention. SDSPis the search pattern for the last step of Diamond Search (DS), whichconsists of four checking points surrounding a current position. In thepreferred embodiment, the proposed MV search process continues with SDSPuntil the point with the minimum cost is at the SDSP center or thenumber of search steps exceeds 3. Setting the maximum number of searchsteps to 3 is to prevent a long search process when the two predictionMVs are close to each other while the best matching point is far away,although this rarely occurs. If it occurs, apply ADSS instead of SDSP.If the current search point overlaps with previous search points, skipthe point.

The number of search points per MB is calculated in the unit of the SADcalculations needed for a search point for the block-size of 16*16.Since the SAD calculation varies for different block-size, a calculationweight is applied to search points of each block-size, which is definedas:

$\begin{matrix}{{SP}_{MB} = {\sum\limits_{i = 1}^{7}\; {W_{i}\left( {\sum\limits_{j = 1}^{Ni}\; {SP}_{ij}} \right)}}} & (4)\end{matrix}$

In Equation (4), SP_(MB) is the number of search points per MB. _(Ni) isthe number of blocks in one MB for block-mode i: _(Ni)=1, 2, 2, 4, 8, 8,16 for 16*16, 16*8, 8*16, 8*8 8*4, 4*8 and 4*4 mode respectively.SP_(ij) the number of search points for the block with mode i and blockserial number j. W_(i) is the calculation complexity weight fordifferent block sizes. W_(i)=1, ½, ½, ¼, ⅛, ⅛ and 1/16 for 16*16, 16*8,8*16, 8*8, 8*4, 4*8, and 4*4 mode respectively.

The search points for the Merge and Split algorithm of the presentinvention come from two processes: one is the ADSS process for the 8*8blocks, the other is from the MV search after the prediction for otherblock-sizes. Generally, the number of ADSS search points is less thanthat by DS, which is at least 13 points. Note that for the splittingfrom 8*8 blocks to 8*4 and 4*8 blocks, the number of MV search points isat least 5 points. For other parts of the merge and split process, thesearch points can be either 0, or at least 5 points.

The simulation results of a preferred embodiment of the presentinvention will be described below. Based on the MV prediction and the MVsearch procedures as mentioned above, three merge/split schemes:bottom-up merge, top-down split, and merge-split, are implemented in thepresent invention. For the bottom-up merge that begins with 4*4 blocks,if the two MVs of the two smaller-size blocks are equal to each other,then return the MV as the MV for the larger block's MV without the MVsearch. Otherwise, it calculates an average MV as the prediction MV andperforms the SDSP search. For the top-down split that begins with 16*16blocks, the MV of larger block is taken as prediction MV and SDSP isalways performed for all smaller blocks.

DS and ADSS are also implemented in the stimulation experiments forcomparisons. The two approaches are applied to seven block-sizesindependently. For DS, the minimum number of search points for eachblock-size is 13. Therefore, for seven block-sizes, the number of searchpoints is 13*7=91. For ADSS, the number of search points is less thanthat of DS, especially for low motion sequences such as the Mother andDaughter sequence.

Further, the simulation conditions are: Frame-rate=30 frames/s, Searchrange=[−16, +15]. Therefore, the comparisons of PSNR, bit rate, andsearch point are shown in Table-4 as below.

TABLE 4 Comparison of PSNR, bit-rate, and search points (SP: SearchPoints) Bit-rate SP per PSNR (kb/s) MB Foreman QCIF, QP = 30 FFS 34.82161.78 961 DS 34.75 162.18 107.45 ADSS 34.79 161.24 76.43 Merge & Split34.77 163.62 33.62 Bottom-up 34.71 164.45 32.47 Merge Top-down 34.72161.42 35.87 Split Coast Guard CIF, QP = 30 FFS 33.18 1058.39 961 DS33.15 1054.55 104.84 ADSS 33.17 1057.55 103.92 Merge & Split 33.151067.53 39.38 Bottom-up 33.11 1074.73 40.06 Merge Top-down 33.10 1051.0039.74 Split Mother and Daughter CIF, QP = 30 FFS 36.79 393.17 961 DS36.76 394.97 109.40 ADSS 36.78 393.30 64.86 Merge & Split 36.75 396.5928.43 Bottom-up 36.74 400.38 23.96 Merge Top-down 36.75 395.26 32.51Split Mobile CIF, QP = 30 FFS 32.71 2084.64 961 DS 32.69 2073.08 106.97ADSS 32.72 2074.63 94.18 Merge & Split 32.70 2096.26 36.45 Bottom-up32.69 2127.02 36.88 Merge Top-down 32.62 2075.02 36.78 Split

From the above simulation results, it is noted that the proposedmerge-split, bottom-up merge, and top-down split schemes all achievereasonably good results. For consistent motion sequences such as “Motherand Daughter,” usually the MVs of the larger block-sizes are the same asthe MVs of the smaller block-sizes. In these cases, the MV search aroundthe search center is not necessary. However, for the “top-down split”scheme, it always performs the MV search around the search-center forall block-sizes, so it is not as efficient as it could be. Further, thePSNR and the bit-rate are very close to “merge-split”, but the MV searchneeds more calculations. On the contrary, for “bottom-up merge,”consistent motion sequences result in less search points, since forconsistent MVs of smaller blocks, MV search is not necessary for largerblocks.

FIG. 5 illustrates system module 50 for fast variable block-size ME inaccordance with the present invention. As shown, sub-module 501 firstperforms a motion estimation for a n×m block of a video image. Asdescribed above, for a 16*16 block, it first take the 8*8 block as theinitial block-size for MV predictions since the 8*8 block size isrelatively close to 16*16 and 4*4 block sizes. Therefore, sub-module 501processes the 8*8 block to generate four prediction MVs by means of theADSS or any other suitable method. The ADSS has been described abovewith reference to FIG. 1. The predicted MVs of the 8*8 block will belater used for prediction in merging and split processes. System module50 further comprises memory sub-modules 504 and 505 that providereference frame and current frame image information, respectively, tothe system.

Merging sub-module 502 next receives the MVs from MV predictionsub-module 101 and performs a merging process on the 8*8 block. Themerging process has been described above in FIG. 1 and thus no detailsneed to be described here. Basically, the 8*8 block is merged to 8*16,16*8, and 16*16 blocks. Sub-module 502 uses the received MVs to predictMVs of the three larger blocks and outputs the predicted MVs to minimumRD cost measurement sub-module 506.

The output of MV prediction sub-module 501 can be further applied tosplitting sub-module 503 for performing a splitting process. Similarly,the splitting process has been described above in FIG. 1 and thus nodetails need to be described here. In summary, the 8*8 blocks are splitinto 4*8, 8*4, and 4*4 blocks. Sub-module 503 take the MVs of the 8*8block processed in sub-module 501 as the prediction MVs and performs theSDSP search. If two MVs of the corresponding 8*4, and 4*8 blocks equalto each other, then return the MV as the MV of the 4*4 block. Otherwise,the MVs are averaged and the averaged MV is taken as the MV searchcenter. The MV output by splitting sub-module 503 is also applied tominimum RD cost measurement module 506 for further processing.

In accordance with the present invention, the merging process and thesplitting process are not necessarily performed at the same time. Forexample, in a bottom-up embodiment wherein n and m both equal to 4, onlymerging process is necessary. In this case, merging sub-module 502merges the 4*4 blocks into 4*8, 8*4, 8*8, 8*16, 16*8, and 16*16 blocks.In the contrary, in a top-down embodiment wherein n and m both equal to16, no merging process is necessary. In this case, only splittingsub-module 503 is required to split the 16*16 block into 16*8, 8*16,8*8, 8*4, 4*8, and 4*4 blocks.

Finally, minimum RD (Rate Distortion) measurement sub-module 506integrates all MVs and prediction errors generated by merging sub-module502 and splitting sub-module 503 to calculate a minimum Lagrangian costby using the MVs as Rate and prediction errors as Distortion. As theLagrangian cost measurement is conventional and is not within the realmof the present invention, its description is omitted here. After theprocess of sub-module 506, a Block Mode and MVs of the images aredetermined.

The foregoing disclosure of the preferred embodiments of the presentinvention has been presented for purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Many variations andmodifications of the embodiments described herein will be apparent toone of ordinary skill in the art in light of the above disclosure. Thescope of the invention is to be defined only by the claims appendedhereto, and by their equivalents.

Further, in describing representative embodiments of the presentinvention, the specification may have presented the method and/orprocess of the present invention as a particular sequence of steps.However, to the extent that the method or process does not rely on theparticular order of steps set forth herein, the method or process shouldnot be limited to the particular sequence of steps described. As one ofordinary skill in the art would appreciate, other sequences of steps maybe possible. Therefore, the particular order of the steps set forth inthe specification should not be construed as limitations on the claims.In addition, the claims directed to the method and/or process of thepresent invention should not be limited to the performance of theirsteps in the order written, and one skilled in the art can readilyappreciate that the sequences may be varied and still remain within thespirit and scope of the present invention.

1. A system module for fast motion estimation of variable block sizes of a video image frame, comprising: a first motion estimation sub-module for processing an initial n×m block to obtain a plurality of prediction MVs; and a merging sub-module for merging the n×m block into larger-size blocks and for receiving the plurality of prediction MVs from the first motion estimation sub-module, the merging sub-module using the plurality of prediction MVs to calculate prediction MVs for the larger-size blocks.
 2. The system of claim 1, further comprising a splitting sub-module for splitting the n×m block into smaller-size blocks and for receiving the plurality of prediction MVs from the first motion estimation sub-module, the splitting sub-module using the plurality of prediction MVs to calculate prediction MVs for the smaller-size blocks.
 3. The system module of claim 1, further comprising at least one memory that provides a reference image frame and a current image frame to the first motion estimation sub-module for processing.
 4. The system module of claim 1, further comprising a minimum cost measurement sub-module, the minimum cost measurement sub-module receiving the prediction MVs for the larger-size blocks, calculating a smallest Lagrangian cost from the prediction MVs, and determining a preferred Block Mode and corresponding MVs.
 5. The system module of claim 1, wherein the merging sub-module utilizes an Adaptive Diversity Search Strategy (ADSS) to calculate the prediction MVs for the larger-size blocks.
 6. The system module of claim 2, wherein the splitting sub-module utilizes an Adaptive Diversity Search Strategy (ADSS) to calculate the prediction MVs for the smaller-size blocks.
 7. The system module of claim 1, wherein n=8, and m=8.
 8. The system module of claim 1, wherein the merging sub-module does not function when n=16, and m=16.
 9. The system module of claim 2, wherein the splitting sub-module does not function when n=4 and m=4.
 10. The system of claim 1, further comprising a sub-module for conducting a MV search with Small Diamond Search Pattern (SDSP) on the prediction MVs obtained by the merging sub-module.
 11. A system module for fast motion estimation of variable block sizes of a video image frame, comprising: a first motion estimation sub-module for processing an initial n×m block to obtain a plurality of prediction MVs; and a splitting sub-module for splitting the n×m block into smaller-size blocks and for receiving the plurality of prediction MVs from the first motion estimation sub-module, the splitting sub-module using the plurality of prediction MVs to calculate prediction MVs for the smaller-size blocks.
 12. The system of claim 11, further comprising a merging sub-module for merging the n×m block into larger-size blocks and for receiving the plurality of prediction MVs from the first motion estimation sub-module, the merging sub-module using the plurality of prediction MVs to calculate prediction MVs for the larger-size blocks.
 13. The system module of claim 11, further comprising at least one memory that provides a reference image frame and a current image frame to the first motion estimation sub-module for processing.
 14. The system module of claim 11, further comprising a minimum cost measurement sub-module, the minimum cost measurement sub-module receiving the prediction MVs for the smaller-size blocks, calculating a smallest Lagrangian cost from the prediction MVs, and determining a preferred Block Mode and corresponding Mvs.
 15. The system module of claim 11, wherein the splitting sub-module utilizes an Adaptive Diversity Search Strategy (ADSS) to calculate the prediction MVs for the smaller-size blocks.
 16. The system module of claim 12, wherein the merging sub-module utilizes an Adaptive Diversity Search Strategy (ADSS) to calculate the prediction MVs for the larger-size blocks.
 17. The system module of claim 11, wherein n=8, and m=8.
 18. The system of claim 11, further comprising a sub-module for conducting a MV search with Small Diamond Search Pattern (SDSP) on the prediction MVs obtained by the splitting sub-module. 